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Abstract 

We  present  an  implementable  feasible  direction  subgradient  algorithm 
for  minimizing  the  maximum  of  a finite  collection  of  functions  subject  to 
constraints.  It  is  assumed  that  each  function  involved  in  defining  the 
objective  function  is  the  sum  of  a finite  collection  of  basic  convex  func- 
tions  and  that  the  number  of  different  subgradient  sets  associated  with 
nondif ferentiable  points  of  each  basic  function  is  finite  on  any  bounded 
set.  It  is  demonstrated  that  under  certain  conditions,  including  continuous 
differentiability  of  the  constraints  and  a regularity  condition  of  the 
feasible  region,  that  the  algorithm  generates  a feasible  sequence  which 
converges  to  an  £-optimal  solution. 

The  results  of  some  computational  experiments  are  included. 
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1 . Introduction 

In  this  paper,  we  develop  an  implementable  feasible  direction 
algorithm  for  solving  a class  of  nondifferentiable  nonlinear  programming 
problems  of  the  form: 

(P)  Min  F(x) , 

X £ C 

where 

F(x)  = max  (fi(x)  = lj=1  f^Cx)  ; i = 1,  m}, 

f^  finite,  convex,  not  necessarily  differentiable, 

C = {x  £ Rn  ; H(x)  < 0}, 

H(x)  = max  [h^x)  ; i = 1,  . ..,  K], 
h.(x)  convex  on  Rn. 

l 

Our  present  work  is  related  to  an  earlier  paper  [1]  where  we  presented 
an  algorithm  for  solving  an  unconstrained  version  of  problem  (P). 

Our  approach  follows  that  of  Dem'yanov  and  Malozemov  [5]  who  employ 

| l 

t 

subgradients  to  devise  a descent  algorithm  for  minimizing  the  maximum  of 
continuously  differentiable  convex  functions,  i.e.,  problem  (P)  with  all  f^ 
differentiable  and  C = Rn.  For  related  literature  on  nondifferentiable 
optimization  methods,  the  surveys  by  Mifflin  [13,  14]  are  recommended. 

He  traces,  e.g.,  the  development  of  heuristic  methods  by  Held  and  Karp  [8] 

and  Held,  Wolfe  and  Crowder  [9];  the  convergent  methods  of  Polyak  [15]  i 

and  Bertsekas  and  Mitter  [4];  and  the  conjugate-type  methods  of  Lemarechal 

[10]  and  Wolfe  [17].  Mifflin's  algorithm  [13]  for  problems  with  "weakly 

upper  semismooth"  functions  [ 14 ] builds  on  the  notion  of  generalized  gradients 

introduced  by  Clarke  [2]  for  Lipschitz  functions.  While  (P)  is  a special 

case  of  that  problem  type,  the  algorithm  developed  here  differs  in  that 

it  is  a feasible  direction  method. 
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Much  of  the  notation  in  this  paper  is  similar  to  the  notation  in  [ 1 ] . 
Denote  n dimensional  Euclidean  space  as  Ru  and  IMI^  as  the  £^-norm  of 
x £ Rn,  where  ||x||  is  the  £2~norm.  Given  a point  x £ Rn,  the  Euclidean 

ball  about  x of  radius  r)  is  N(x,  r))  ; when  x = 0 and  i]  = 1,  B = N(0,  1)  is 

the  Euclidean  unit  ball.  For  a function  f defined  on  Rn  let  3f(x)  be 
the  subgradient  set  of  f at  x and  let  f'(x,  d)  be  the  directional  derivative 
of  f at  x in  the  direction  d.  Given  a set  S c Rn,  Conv  (S)  is  the  convex 
hull  of  S and  Nr(S)  is  the  element  of  minimum  Euclidean  norm  in  S.  Also 
denote  3f(S)  s u (3f(x)  ; x £ S}. 

For  convenience,  the  functions  f^  in  problem  (P)  are  assumed  to  be 
the  sum  of  exactly  £ functions  f^.,  where  perhaps  for  some  i,  some  of  the 
f^j  functions  are  identically  zero  on  Rn.  Clearly,  both  F and  H are 
continuous,  finite,  convex  functions  on  Rn. 

Given  £ > 0 and  p > 0,  at  any  x £ Rn  we  define 

R(x,  £)  = {i  £ { 1,  2,  ...,  m}  ; f.(x)  > F(x)  - £}, 


Q(x,  p)  = {i  £ { 1 , 2,  ...,  K}  ; 0 > h.(x)  > - p}. 

With  these  definitions,  we  can  interpret  f^,  i £ R(x,  e),  as  an  "e-binding" 
objective  function  af  x;  and  h^ , i £ Q(x,  p),  as  a "p-binding"  constraint 
at  x.  When  H(x)  < -p,  we  say  that  x is  a p-feasible  point  and  the 
set  of  all  x in  Rn  where  H(x)  < -p  is  the  p-feasible  set. 

Given  x £ Rn  and  r)  > 0,  we  define 

G^(x,  r|)  = { x } u {y  ; y £ N(x,  y),  f ^ . not  differentiable  at  y}, 
Si(x,  0)  = Ij.j  3fij  (G^j  (x , r))),  i = 1,  2,  ...,  m. 

In  addition,  let 

(x , £,  n)  = u {s.(x,  n)  ; i £ R(x , £)). 

To  handle  constraints,  we  use  a procedure  somewhat  similar  to  that 
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presented  in  [13].  At  any  x £ Rn,  we  consider  subgradients  of  p - binding 
constraints  by  defining  S (x,  (j)  = u{3h^(x)  ; i £ Q(x,  p)}  and  letting 
S(x,  £,  p,  n)  = Conv  [S1  (x,  £ , r|)  u S2(x,  p)}. 

We  note  that  for  an  unconstrained  problem,  S(x,  £,  r),  p)  = S*(x,  £,  r|) 
is  precisely  the  enlargement  of  the  subgradient  set  considered  in  [ 1 ] . 

We  assume  the  functions  f„  are  LFS  (locally  finitely  subdifferentiable) , 
which  means  that  in  any  closed  bounded  Euclidean  ball,  the  number  of  different 
subgradient  sets  of  f^.  corresponding  to  the  points  of  nondifferentiability, 
is  finite.  In  [1],  we  cite  several  examples  of  LFS  functions,  including 
examples  from  location  theory  and  linear  approximation  problems. 

Associated  with  S(x,  £,  p,  r)),  the  function  41  measures  the 
proximity  of  S(*)  to  zero: 

Y(x,  £,  p,  r|)  = min  (max  {(g,  d)  ; d £ S(x,  £,  p,  n)l  ; g £ B}. 

It  is  easily  established  that 

YU,  £ , p,  n)  = - | |Nr(S(x,  £ , p,  n) ) | | - 
We  note  that  4*  is  well  defined  since  S is  a nonempty,  compact,  convex  subset 
of  Ra.  Further  4*  is  always  nonpositive.  When  4*(x,  £,  p,  r|)  = 0>  we  call 
x a stationary  point,  and  any  x where  4,(x,  £,  p,  r|)  < 0 is  a nonstationary 
point.  In  the  next  section  we  consider  properties  of  stationary  and 
nonstationary  points. 

2 . Stationary  and  Nonstationary  Points 

In  the  unconstrained  problem,  information  given  by  the  value  of  4*  is 
relatively  easy  to  exploit  [1]  since  the  set  S(x,  £,  r|)  is  derived  solely 
from  the  f^  functions  which  are  £-binding.  In  the  constrained  problem 
this  is  no  longer  the  case,  since,  to  construct  S(x,  £,  p,  r|)>  we  also 
consider  the  subgradient  sets  of  the  p-binding  constraints.  Consequently, 
stationarity  in  the  constrained  problem  will  not  always  imply  a lower 
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bound  on  the  minimum  value  of  F on  C,  F*,  as  is  the  case  with  the 
unconstrained  problem.  In  the  presence  of  constraints,  we  show  in  the 
next  theorem  that,  while  it  may  be  possible  to  obtain  a lower  bound  on 
F*,  it  may  well  be  the  case  that  the  only  implication  of  stationarity  is 
the  emptiness  of  the  interior  of  the  p-feasible  set. 

Before  proving  this  theorem,  we  assume  that  the  minimum,  F*,  of  F(x) 
over  C exists.  In  addition,  we  assume  that  there  exists  an  upper  bound, 

6,  on  the  norm  of  any  subgradient  of  any  function  f.^  or  h^  at  any  point 
in  C. 

Theorem  2 . 1 Let  x £ C be  a stationary  point. 

a)  If  H(x)  < -p,  then 

F(x)  > F*  > F(x)  - £ - 2Jln6-  (2.1) 

b)  If  -p  < II ( x ) < 0,  at  least  one  of  the  following  is  true: 

bl)  (2.1)  holds , 

b2)  The  interior  of  the  p-feasible  set  is  empty, 

b3)  F(z)  > F(x)  - £ - 2£r|6,  for  all  p-feasible  z. 

Proof : Stationarity  at  x is  equivalent  to  0 £ S(x,  £,  p,  n),  which  can 

occur  if  and  only  if  there  exists  gj,  ...,  g^,  where 
1 2 

gi  £ (S  (x,  £ , q)  u 5 (x,  p)}  and  0 £ Conv  (g} , ....  g ).  Index  the 
g^  so  that  g^  £ S^(  x,  £ , r|)  for  1 < i < and  g.  £ S^(x,  p)  for  + 1 < i < q 

Thus  for  i,  1 < i < q ^ , g^  £ 3f^^(y^),  where  f^.-j(x)  > F(x)  - £ and 

| |x  - y.|  | < n;  and  for  i,  qi  + 1 < i < q,  gd  £ 3 h^.^x),  where 

0 ^ h^jOO  = ~V- 

We  note  in  the  case  where  q^  = q (which  certainly  holds  in  case  a)), 

that  g^  £ S 1 (x , £ , r|) , 1 < i < q,  and  thus  by  Theorem  3.2  of  [1],  (2.1) 

holds.  Thus  suppose  q^  < q,  in  which  case  0 = q^  or  0 < q^ 
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If  0 = q , then  g.  £ S (x,  p)  for  aH  1-  By  the  subgradient  inequality, 
k(i)(z)  = h^(x)  + ^i  > z-x)  v z £ Rn,  and  further  h^.^(x)  > -|J  and 
H(z)  > h^.^(z).  Thus 

H(z)  > h^(z)  > -p  + (gi}  z-x)  v z e Rn. 

Taking  the  convex  combination  over  all  i = 1,  q,  yields  H(z)  > -p , 

v z £ Rn,  and  hence  the  p-feasible  set  has  an  empty  interior,  which 
establishes  case  b2). 

Consider  the  remaining  case,  1 < q^  < q.  As  in  [l],  since  each 
function  f^  is  Lipschitz, 

F(z)  > F(x)  - £ + (g^,  z-x)  - 2£q6  , v z £ Rn,  1 < i < q^;  and 
thus  for  all  z £ Rn, 

F(z)  > F(x)  - £ - 2£q6  + max{(gi,  z-x)  : i = 1,  ...,  q1).  (2.2) 

For  any  p-feasible  z,  and  i = q1  + 1,  ...,  q, 

"P  > h^(z)  > h^.^(x)  + (gi?  z-x)  > -p  + (gi,  z-x), 

which  implies  (g^,  z-x)  < 0.  Writing  zero  as  a convex  combination  of 

all  g^  leads  to  max{(g^,  z-x)  : i = 1,  ...,  q^}  > 0 for  all  p-feasible  z. 

Thus  the  final  term  of  (2.2)  may  be  deleted  and  case  b3)  is  established. 

Remark  2 . 1 The  definition  of  q^  in  the  previous  proof  implies  that  if 

1 2 

a point  g^  is  in  both  sets  S (x,  £,  q)  and  S (x,  p),  it  should  be 
considered  as  a point  of  S*(x,  £,  q).  This  forces  q^  to  be  equal  to 
q if  possible  and  thus  to  obtain  the  more  useful  cases  (a)  or  (bl)  in 
which  we  have  bounds  on  the  constrained  minimum  of  F. 

Remark  2.2  Cases  (bl),  (b2),  and  (b3)  are  not  mutually  exclusive.  When 

several  different  convex  combinations  of  elements  in  S*(x,  £,  q)  and 
2 

S (x,  p)  are  0,  (bl),  (b2),  and  (b3)  can  occur  simultaneously. 

To  illustrate  the  different  situations  described  in  Theorem  2.1  we 
consider  the  following  example. 

j 
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subject  to 


hj (x)  = | |x|  \l  - 1.6  < 0, 

h2(x)  = | |x  - (1,  -1)  | jx  - 1.6  < 0, 

Let  £ = 0,  r|  = -Jl  x 10  * . It  is  clear  that  6 = and  x*  = (.5,  -.5) 
is  the  optimal  solution  to  this  problem.  For  different  values  of  p, 
stationarity  may  have  distinct  implications. 

(a)  Let  p = 0.  The  point  y^  = (.6,  -.4)  is  feasible,  with  HCy^) 

= Max{-.6,  -.6}  = -.6  < -p.  y^  is  also  stationary  because  (.5,  -.5) 
is  in  N(Yq,  n)-  We  observe  that  F(y^)  = -jl  x 10  1 , F(Vq)  - 2£r|S 

= -.2588  and  we  obtain  case  (a)  of  Theorem  2.1,  that  is, 

y[2  x 10"1  > 0 = F*  = Min{F (z)  : z £ C}  > -.2588. 

(b)  Let  p = 0.6.  y^  remains  stationary  with  HCy^)  = -.6  = -p. 

O 

In  this  case,  in  addition  to  the  lower  bound  on  F*,  {z  £ : H(z)  < -.6} 

is  empty,  and  both  (bl)  and  (b2)  are  obtained.  On  the  other  hand, 
observe  that  y^  = (1,  0)  is  stationary  since  H(y^)  = h^(y^)  = h^y^) 

= -.6  = -p  so  that  both  points  (1,  -1)  £ Oh^Cy^)  and  (-1,  1)  £ Bt^Cy^) 
are  in  S(yp  £,  p,  r|)  • However,  the  only  conclusion  is  that  the  set 
{z  £ : H(z)  < -.6}  is  empty. 

Now  let  p = 0.6,  and  add  a third  constraint,  h^Cx)  = - x^  - x2  < 0, 
to  Example  2.1.  Consider  y2  = (.8,  -.2).  Since  H(y0)  = Max{-.6,  -.6,  -.6}  = -.6 
= -p,  y2  is  feasible.  Moreover  y2  is  stationary.  It  is  easy  to  verify  that 
S(y2,  £,  p,  r|)  is  the  square  with  vertices  (-1,  1),  (1,  -1),  (-1,  -1),  (1,  1). 
Here  the  lower  bound  obtained  is  only  valid  on  the  p- feasible  set,  which  is  t by- 
line segment  with  end  points  y2  and  (1,  0).  With  F(y0)  = 3<j2  x 10  \ this 
lower  bound  is  positive  and  equal  to  .0242.  Hence,  at  y both  (b2)  and 
(b3)  hold. 
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Remark  2.3  It  is  clear  that  case  (b2)  of  Theorem  2.1  is  relatively 
undesirable  since  nothing  can  be  said  about  the  objective  function  itself. 

To  discard  this  case,  a constraint  qualification,  similar  to  Slater's 
constraint  qualification  [12]  is  necessary.  Thus,  if  the  set 
{z  £ Rn  : H(z)  < -p]  is  nonempty,  case  (b2)  cannot  occur. 

Having  dealt  with  stationary  points  in  Theorem  2.1,  we  now  consider 
nonstationary  points.  Given  a nonstationary  point  x,  the  subgradient 
sets  in  S(x,  £,  p,  r|)  relative  to  the  functions  f^  ensure  that  we  can 
find  a descent  direction,  and  the  subgradient  sets  3h^,  if  any,  ensure 
that  this  descent  direction  is  feasible. 

Theorem  2.2  If  ^(x,  £,  p,  rj)  < 0>  there  exists  a feasible  descent  direction 
for  F at  X. 

Proof:  Let  gQ  ^0  be  the  element  of  minimum  norm  in  S(x,  £,  p,  rj) » i-e., 

0 > f(x,  £ , p,  n)  = -Il80ll  = - 1 |Nr(S(x,  £ , p,  n) ) I I - 
Define  dQ  = -gQ/  | | gQ | ] . If  H(x)  = 0,  then  for  any  i such  that 
h^Cx)  = H(x)  = 0,  3h^(x)  c S(x,  £,  p,  r|)  so  that 
h^(x,  dQ)  = max  {(g,  dQ)  ; g £ 3h.(x)} 

< max] (g , dQ),  g £ S(x,  £,  p,  r|) } 

= -min{ (g,  -dQ),  g £ S(x,  £,  p,  n)l 
= -1/|  |g0|  | min{(g,  gQ),  g £ S(x,  £,  p,  q) ) 

= -||g0ll  < o. 

Hence,  d is  a feasible  direction  at  x.  On  the  other  hand,  if  H(x)  < 0, 
o 

the  direction  d is  feasible  since  the  functions  h.  are  continuous, 
o 1 

In  either  case  above,  since  3F(x)  c S(x,  £,  p,  r|)  it  follows  from 

Theorem  3.3  of  [1]  that  F'(x,  d ) < 0.  Therefore  d is  a descent  direction 

o o 

for  F at  x. 
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3.  The  Algorithm 

The  algorithm  exploits  the  results  of  the  previous  section.  We  choose 
positive  values  for  the  three  parameters  £,  (J,  and  r|.  Let  xq  be  a feasible 
starting  point,  set  k = 0 and  go  to  Step  1. 

Step  1 

At  xk,  find  F(xk),  R(xk,  e),  and  Q(xk>  (j).  Calculate  S(xk,  e,  p,  r|) 
and  ^(xk,  £,  p,  q).  Go  to  Step  2. 

Step  2 

If  £>  M > I)  = G>  stop;  xk  is  a stationary  point. 

If  ^(xk>  £,  p,  q)  = 0 < 0»  define  gk  as  the  element  of  minimum  norm  in 
S(x,  £ , p,  r))  and  let  dk  = -gk/||gk||.  Perform  a restricted  line  search 
along  dk>  finding  tk  such  that 

F(xk  + tkdk)  = Min{F(xk  + tdfe)  ; t > 0,  H(xk  + tdfc)  < 0}. 

Update  xk  ->  xk+J  = xk  + tkdk,  k -»•  k + 1 and  return  to  Step  1. 

In  the  next  section,  under  supplementary  assumptions,  we  prove  that 
limit  points  of  the  algorithm  are  stationary  points  for  problem  (P). 

4.  Proof  of  Convergence 

In  this  section  we  assume  that  the  algorithm  does  not  stop,  but 

generates  an  infinite  sequence  {xk}  converging  to  some  limit  x.v.  The 

proof  that  x,.,  is  stationary  depends  on  the  approximation  of  S(x.v,  £,  p,  q) 
by  S(xk,  £ , p,  q).  Although  the  S(xk>  £,  p,  r|) > d°  not  necessarily  converge 
to  S(x*,  £ , p,  q),  for  sufficiently  large  k,  S(xk,  £,  p,  q)  is  contained 
in  S(xj.,  £ , p,  q)  plus  an  epsilon  ball.  In  addition  to  the  assumption 
that  the  functions  f^  are  LFS,  two  additional  assumptions  are  required  in 
the  proofs: 

Assumption  4.1  There  exists  some  xq  £ C,  a starting  point  for  the 
algorithm,  such  that  the  intersection,  X,  of  C with  the  level  set 
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[x  £ Rn  : F(x)  < F(xq)}  is  nonempty  and  bounded.  By  continuity  of  F and 
H,  X is  also  closed. 

Assumption  4.2  The  constraint  functions,  h^,  i = 1,  2,  ...K  are 
continuously  differentiable  as  well  as  convex. 

The  first  assumption  guarantees  a solution  point  x for  (P)  and 
insures  that  a limit  point  x*  exists.  The  assumption  that  the  h^  are 
continuously  differentiable  is  required  for  the  stationarity  proof, 
especially  Lemma  4.5.  In  the  next  section  we  show  by  counterexample 
that  in  the  absence  of  this  assumption  convergence  to  a nonstationary 
point  is  possible. 

Lemma  4. 1 For  k sufficiently  large, 

Q(xk,  p)  c Q(x*,  p). 

Proof : Follows  from  Assumption  4.2. 

We  now  show  that  S(xk,  £,  p,  q)  approximates  the  set  S(x,,.,  e,  p q). 

In  the  proof  of  this  result,  we  use  Theorem  5.2  of  [1]. 

Lemma  4.2  For  any  y > 0,  there  exists  such  that 

S(xk,  e,  p,  r|)  c S(x*,  £,  p,  q)  + yB,  k > Nr 
Proof:  By  definition, 

S(xk,  £ , p,  q)  = Conv  (S1 (xk,  £ , q)  u S2(xk,  p)).  (4.1) 

Since  S^(xk>  £,  q)  is  identical  to  S(xk>  £,  q)  in  [1],  by  Theorem 

5.2  of  [l],  there  exists  such  that 

S1(xk,  £ , q)  c S^Xju,  £,  q)  + yb  > k > Nj.  (4.2) 

2 

Now  consider  the  sets  S (•)  in  (4.1).  From  Corollary  24.5.1 
of  [16],  since  the  functions  h^  are  assumed  continuously  differentiable, 
for  each  i £ Q(xj.,  p),  there  exists  L^  such  that 

3h.(x.)  c 3h.(x.J  + yB  , k > L..  (4.3) 

X K X t%  X 

It  follows  from  the  definition  of  S2(*)»  (4-3)  and  Lemma  4.1  that  there 


exists  N”  such  that 
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(4.4) 


S2(xk,  m)  c S2(xj.,  H)  + vB  , k > N". 

Letting  Nj  = max{N^,  N^}  and  using  (4.2),  (4.3)  and  (4.4),  the  result 
follows . 

Corollary  4.3  If  4*(x.,.,  e,  p,  n)  < " 2y  < 0,  then  f(xk,  e,  (J,  n)  < " Y>  k > Nj. 
Proof:  See  Corollary  5.3  of  [1]. 

The  following  lemma  is  a partial  converse  of  Lemma  4.2.  It  shows  that 
any  subgradient  of  any  binding  function  at  x,v  can  be  approximated  by  an 
element  of  S(xk>  £,  p,  r))  for  large  k. 

Lemma  4.4  Choose  any  a > 0 and  let  s £ 3 f^(x.v)  ; i £ R(x.v,  0).  Then  for 
k larger  than  some  ^ there  exists  s'  £ S(xk,  £,  p,  r|)  and  t such  that 
s = s'  + t , j |t|  j < a. 

Proof : The  proof  follows  from  the  proof  of  Lemma  5.4  of  [l]  upon  noting 
that  s'  £ S(xk,  £ , r|)  c s(xk>  e>  Mi  n)- 

Lemma  4.5.  For  any  k greater  than  the  of  Corollary  4.3,  there  exists 
some  T > 0,  independent  of  k,  where  with  dk  as  chosen  in  Step  2 of  the 
algorithm, 

H(xk  + t dk)  < 0,  and 

F (xk+i ) i F(xk  + t dk)>  for  all  t £ [0,  Tj. 

Proof : By  Assumption  4.2,  the  tr  are  continuously  differentiable  on  Rn  and 
thus  from  Remark  2 of  [5,  p.  270],  for  any  i = 1,  ...,  k,  there  exists 
T . > 0 such  that  for  all  t £ [0,  T .], 

h^(x  + td)  = h^(x)  + t(Vh^(x),  d)  + cr(x,  d ; t),  (4.5) 

where  cr  is  a function  with  the  property  that  cr(x,  d ; t)/t  •*  0 
uniformly  in  x £ X and  d,  | ]d  | ] = 1 , as  t -*■  0 . Because  of  the 
uniform  convergence  of  cr(x,  d ; t)/t,  we  can  choose  T^,  0 < T^  < Tq^, 
such  that 

t £ [0,  T^]  implies  [(^(x,  d ; t)  j < y t/2,  (4.6) 


n 

I 

for  all  x £ X and  all  d where  | |d|  | = 1. 

Letting  h minjT^.  ; i = 1,  K},  it  follows  from  (4.5)  and 

(4.6)  that  for  any  i = 1,  . ..,  K,  any  x £ X and  any  d,  ||d||  = 1, 
if  t £ [0,  'T  ] , then 

hf(x  + td)  < h.(x)  + t(Vhi(x),  d)  + y t/2.  (4.7) 

At  a point  x^,  for  any  i = 1,  . ..,  K,  either  i £ Q(xk>  p)  or  not. 

If  i £ Q(xk,  p),  then  with  h^(xk)  < 0 and  choosing  d = d^  in  (4.7), 

hi^X  + t(V  = dk^  + ^ 1 11'  (4.8) 

Noting  that  Vh^x^)  £ S(xk,  £,  |j,  r|)  > with  k > N^,  and  from 
Corollary  4.3  that  (Vh.(xk),  gfc)  < Y(xk,  £,  p,  n)  £ "Y » gives, 
from  (4.8), 

h.(xk  + tdk)  < -yt  + yt/2  < 0.  (4.9) 

Consider  any  i £ Q(x,  , p).  By  uniform  continuity  of  the  functions 
h^  on  the  compact  set  C,  there  exists  some  > 0 such  that  for  all 
x,  y in  C,  ] |x  - y | [ <.  T”  implies 

Ih^x)  - h^(y)  | < p for  all  i = 1,  . . . , K.  (4.10) 

For  any  i t Q(xk,  M)»  h^(*k)  < and  so  ^rom  • 10 ) » with 
t £ [0,  r], 

h£(xk  + tdk)  < h.(xk)  + p < -p  + p = 0.  (4.11) 

Letting  T = min  (T^,  T"}  > 0,  from  (4.9)  and  (4.11)  and  the  definition 
of  H, 

H(xk  + tdfc)  < 0 , t £ [0,  T). 

The  second  conclusion  of  the  Lemma  follows  since  Step  2 of  the  algorithm 

determines  x.  where 
k+1 

F(xk+i)  = min{F(xk  + tdk)  ; t > 0,  H(xk  + tdfc)  < 0} 

< min{F(xk  + tdfc)  ; t £ [0,  T)}. 
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In  the  convergence  proof,  we  make  use  of  the  following  result  from 


Cullum,  Donath  and  Wolfe  [3]. 

Lemma  4.6  Let  F be  convex  on  Rn  and  let  the  sequences  and  {d^}  satisfy 

xfc  ■*  x*,  dk  -*  d.v,  and  F(xk+1)  < F(xk  + tdk)  , 0 < t < T.  Then  F'(x,v,  d.,J  > 0. 

We  now  prove  the  main  result  of  this  section  that  the  limit  point  of  any 
convergent  sequence  generated  by  the  algorithm  is  a stationary  point. 

Theorem  4.7  <Kx,v,  £,  p,  r|)  = 0. 

Proof:  Suppose,  to  the  contrary,  that  ^(x,,.,  £,  p,  r|)  < *2y  < 0.  Since 


d,  ■+  d...  £ B there  exists  N~  such  that 

k 3 

£6lldk  - d*||  < |,  k > N3.  (4. 

The  directional  derivative  of  F at  xy.  is 

F’  (xjl,  d.v)  = Max{ (g,  dj.)  ; g £ 3F(x.v)}  = (IX^,  d,  J , 

where  IX.  s.  is  a convex  combination  of  elements  s.  and  each  s.  £ 3f.(x...) 

J J J J l -’ 

for  some  i £ R(x.£,  0).  Choose  a single  so  large  that  Lemma  4.4  holds 
for  all  such  s^  with  a = |.  Let  k > MaxfNj,  N3,  N^},  where  Nj  is  from 
Lemma  4.2.  Employing  Lemma  4.4, 

F'(x*,  d*)  = (IX.s',  d*)  + (IX. t.,  d*) 

= (IXjSj,  dk)  + (IXjSj,  d*  ‘ dk)  + 

< (IX.s’,  dk)  + (IX.s’.d*  - dk)  + ||lX.t.  M !|d*|| 

< (IX  sj,  dk)  + (IXjSj,  d*  - dk)  + 

By  definition  of  dk  and  from  Corollary  4.3, 

(2Xj  sj,  dk)  < Max{ (g,  dk)  ; g £ S(xk,  £,  p,  r|)l 
= 1,(xk,  £ , p,  n)  < “Y  • 


(4.12) 


Further,  ! Isj  II  i &&  and  from  (4.12), 

ox  s'  d*  - d ) < ||IX  s!  | 
J J K J J 

Combining  these  results  gives 
F'(x*,  d*)  < "|  < 0 , 


Id*  - dkM  < l 
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which  contradicts  Lemma  4.6  (with  T as  in  Lemma  4.5).  Therefore, 

Y(x*,  £ , M>  n)  = o. 

When  executed  on  a computer,  the  result  of  the  difference  F(x)  - £, 
appearing  in  the  definition  of  R(x,  e)  is  not  very  different  from  F(x) 
if  F(x)  is  a large  number  since  in  general  e is  small.  Consequently, 

R(x,  £)  might  be  reduced  to  R(x,  0)  through  roundoff  error  and 
this,  in  turn,  could  affect  the  convergence  of  the  algorithm. 

To  avoid  this  numerical  problem,  redefine  R(x,  £),  as  is  done  in  [l], 
and  use  instead 

R'(x,  e)  = {i  = 1,  2,  ...,  m ; f^(x)  > F(x)  - £ F(x)}. 

It  is  also’ necessary  to  assume  F*  > 0 and  to  modify  the  definition  of 
S^(x,  £,  r|)  given  in  Section  1.  Then  the  results  of  Sections  2 and  4 
hold  with  the  exception  that  the  inequalities  (2.1)  are  modified  to  be 
(1  - £)F(x)  - 2£r]6  < F*  < F(x) . 

5 . Computational  Results  and  a Non  Convergent  Example 

In  this  section,  numerical  results  are  presented  for  several 
constrained  minisum  location  problems  and  a constrained  minimax  location 
problem.  To  find  the  point  of  minimum  norm  in  S(x,  £,  p,  r|)>  Wolfe's 
algorithm  [18]  was  used  for  minimax  problems,  and  Gilbert's  algorithm  [6] 
was  used  for  minisum  problems.  The  line  search  was  done  with  quadratic 
fits,  but  was  modified  so  as  to  yield  a feasible  point  at  each  iteration. 
All  the  programs  were  written  in  FORTRAN  and  run  on  an  IBM  370/165. 

The  constrained  minisum  problems  are  from  [11].  Three  new  facilities 
are  to  be  located  in  the  plane  relative  to  five  existing  facilities. 

There  is  a single  linear  constraint  on  the  location  of  the  third  new 
facility,  (x^j,  x^)*  name^y»  x3i  + x32  - 3 < 0. 
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The  problem  to  solve  is 

MinFCXj,  X2>  X ) = I3  £ wrs  | |X  - A$ 


+ ✓q  V I |x  - X J I 

l<r<t<3  rt  1 1 r t 1 1 p 


subject  to: 


X31  + X32  " 3 = °> 

where  = (x^,  xr2)  is  the  location  of  the  r*"*1  new  facility; 

Ag  = (agp  is  the  fixed  location  of  the  s^  existing  facility,  and 

w and  v are  known  positive  weights.  Note  that  the  function  F is 
rs  rt 

the  sura  of  (2  = 18)  LFS  functions  and  that  the  single  constraint  is 
continuously  differentiable. 

From  [11],  Aj  = (2,  3),  A2  = (4,  2),  A3  = (5,  4),  A.  = (3,  5) 
and  A_  = (6,  7),  and  the  w and  v data  are  as  in  Table  1.  The  problem 

j rs  iL 

was  solved  with  three  different  values  for  p (p  = 1,  1.78,  and  2).  In  all 

cases  the  starting  point  was  Xj  = = X^  = (0,  0)  and  p was  10  ^ . For 

the  problem  with  p = 1,  r|  = 10  and  for  the  problems  with  p = 1.78  and  p = 2, 
-3  -5 

both  r|  - 10  and  r)  = 10  were  used.  A summary  of  the  computational  results 
is  given  in  Table  2,  where  the  upper  and  lower  bounds  on  F*  are  as  given 
in  [11]. 

We  remark  that  for  the  problems  with  rj  = 10  ^ and  p = 1.78  and  p = 2, 
termination  occured  when  the  maximum  number  of  iterations  allowed  (150) 
was  reached,  but  progress  was  still  possible.  For  the  other  problems,  the 
algorithm  reached  stationarity . The  comparison  with  the  results  of  [11]  is 
rather  difficult  to  make,  since  no  numbers  of  iterations  or  computing 
times  are  reported  in  that  reference. 

As  a constrained  minimax  location  problem,  the  Caribbean  Islands 
problem  formulated  in  [1],  was  modified  to  include  the  four  constraints 


|X1  - Ajlr  < 144,  1 |X1  - A4 


aJ  I2  < 121,  | |x2  - Aj  I2  < 225,  and 
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| | X2  - A^ll  < 144.  The  points  A^,  i = 1,  2,  3 and  4 represent  the  locations 
of  four  of  the  Caribbean  cities  in  the  problem  (A^  = (11.4,  11.6), 

A2  = (35.3,  13.5),  A3  = (8.80,  37.2),  and  A4  = (20.9,  30.6)). 

“6  *5  *5 

Parameters  were  given  values  £ = 5 x 10  , r)  = 10  and  p = 10  . Starting 
from  (Xj,  X2)  = (x^,  xl2,  x21,  x22)  = (15,  22,  26,  11),  the  algorithm 
terminated  after  15  iterations  and  3.03  seconds  CPU  time  at  the  stationary 
point  (13.902,  23.201,  24.546,  18.824)  with  a function  value  of  28.024. 

-14 

The  value  of  H at  the  stationary  point  was  -1.07  x 10 

With  a fifth  constraint  added,  i !x^  - X2lP  < 118,  and  starting 
from  (19,  20,  24,  16),  the  answer  (14.585,  23.170,  24.488,  18.705)  with 
F = 28.158  and  H = 0 was  obtained  in  24  iterations  and  2.36  seconds  of 
CPU  time. 

We  now  give  an  example  which  shows  that  if  the  constraints  of  problem 

(P)  are  not  continuously  differentiable,  the  sequence  generated  by  the 

algorithm  may  converge  to  a nonstationary  point.  Consider  the  problem: 

Min-  F(X),  F(X)  = -2x„  + x- 
XelT  Z 

subject  to 

tij (X)  = Max{3x3  + x2  ~ ^:<3’  " ^xl  + x2  ” ^X3^  = ® 
h2(X)  s x3  - 1 < 0 , 

which  has  the  optimal  solution  X*  = (0,  2,  1)  with  F(X*)  = -3.  The  functions 

3 

F(x)  and  h2(X)  are  continuously  differentiable  on  R , but  h^(X)  is  not. 

In  fact  hj(X)  is  LFS  with  possible  subgradients  sets  of  {(3,  1,  -2)}, 

{(-3,  1,  -2)}  or  Conv({(3,  1,  -2),  (-3,  1,  -2)}).  To  simplify  notation, 
define  - Conv({(0,  -2,  1),  (3,  1,  -2)}),  and  = Conv(((0,  -2,  1), 

(-3,  1,  -2)}).  It  is  easy  to  verify  that  Nr(^)  = (.1,  -1,  0)  and  Nr  <v2) = 

(-1,  -1,  0).  Since  F is  differentiable,  set  r)  = 0. 
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Also  it  is  clear  that  e = 0 is  a legitimate  choice.  Further,  let 


p = .5  and  let  Xq  = (1,  -3,  0)  be  the  starting  point  for  the  algorithm,  where 
F(Xfl)  = 6. 

Since  Ii^Xq)  = Max{0,  -6}  = 0 and  h2(XQ)  = -1,  Q(Xq,  |j)  = {l}, 
and  S(Xq,  e,  p,  r|)  Thus  the  feasible  descent  direction  given  by 

the  algorithm  is  -Nr ~ (-1,  1,0).  To  find  X^,  minimize 
F(Xq  + t(-l,  1,  0))  = -2t  + 6,  for  t > 0 and  (Xq  + t(-l,  1,  0))  feasible. 

The  minimum  in  the  feasible  set  occurs  for  t = 3/2  which  gives 
Xx  = (-1/2,  -3/2,  0),  F(XX)  = 3. 

At  Xj,  hx(Xx)  = Max { - 3 , 0},  h^Xj)  = -1  and  S (X1 , e,  u,  q)  = 

With  -Nr($/,)  = (1,  1,  0),  minimize  F(X^  + t ( 1 , 1,  0))  over  t > 0 and 
(X^  + t ( 1 , 1,  0))  feasible,  obtaining  t = 3/4  and  X2  = (1/4,  -3/4,  0) 
with  F(X2)  = 3/2. 

X0  yields  S(X2,  £,  p,  q)  = ^ and  the  minimization  in  the  direction 
-Nr(^/j)  gives  X^  = (-1/8,  -3/8,  0),  F(X2)  = 3/4.  Continuing,  it  can  be 
shown,  using  inductive  arguments,  that  for  any  k, 

Xk  = ((-l)k/2k  , - 3/ 2k  , 0)  and  F(Xfc)  = 3/2k-1. 

Consequently,  the  sequence  {X^}  converges  to  X...  = (0,  0,  0),  while 
F(Xk)  ■*  F(X.V)  = 0.  However,  Xv_  is  not  a stationary  point.  This  can  be 
seen  from  the  fact  that  hx(Xj.)  = 0,  h2(X.,.)  = -1,  yielding  Q(Xj.,  p)  = {l}, 
and  S(X*,  e,  M>  q)  = Conv({(0,  -2,  1),  (3,  1,  2),  (-3,  1,  -2)}), 
where  0 £ S(X.V,  £,  p,  q).  Further,  -Nr(S(X*,  £,  p,  q))  = (0,  1/2,  1/2) 
which  is  a feasible  descent  direction  for  F at  X,v. 


p 

1 

1.78 

2 

n 

10"5 

10"3  10'5 

10"3 

IQ'5 

Number  of 
iterations 

63 

90  150 

134 

150 

CPU  time 

6.62s 

14. 39s  14.72s 

16.24s 

13.77s 

Lower  bound  on 

F*  90 

70.25804 

68.18406 

Value  obtained 

90.00008 

70.27462  70.28511 

68.23939 

68.28341 

Upper  bound 
on  F* 

90 

70.4299 

68.64550 

Constraint 

value 

-8 . 17xl0-6 

0 -1.23xl0-5 

0 

0 

Table  2.  Numerical  results  for  the  constrained 
minisum  location  problem. 
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